The software testing phase is a critical phase in the software development process. During the software development process, the software testing phase occurs after the software has been designed, implemented in a programming language, and tested to a limited degree. During the testing phase, software testers test the software extensively to ensure that the software meets all of the requirements it is intended to meet. In order to accommodate simultaneous testing of several different software packages by several testers, multiple test machines are often implemented. Different types of software packages may need to be tested on different types of test machines, such as, for example, test machines with different hardware configurations and/or different operating systems. When a large number of software testers are required to share common resources for software testing, provisions must be made for scheduling the tests in order to efficiently manage these shared resources. The efficient management of these shared resources may also require that tests and the results of the tests be recorded so that the tests can be used repeatedly if needed and so that the results of the tests can be analyzed and subsequently used for comparison with the results of tests performed at a later time.
In an effort to maximize efficiency in the handling of test scheduling and test execution, attempts have been made to automate software testing by using a server to manage test machines and to allocate test packages among the test machines in accordance with a schedule. Generally, these types of systems pre-allocate tasks to test machines by calculating the current and scheduled loads on the test machines and scheduling the tasks so that they are performed in a tine-efficient manner. For example, Sun Microsystems, Inc. has proposed an automated task-based scheduler for use with UNIX platform systems which allows users operating "client" machines to schedule tests to be executed on "target" machines. A central server receives a request from a client machine to perform a task. The server maintains information relating to all currently scheduled tasks on all target machines in a "status" database. The server maintains information relating to the expected duration of each test package and other test package attributes in a "packages" database.
When the server receives a request to perform a task from a client machine, the server determines the loads on each of the target machines which are suitable for performing the task. The loads are determined based on the expected duration of each test package. The server then schedules the task on the target machine with the least current load. A task file created at the client machine and copied to the server includes priority information relating to the task requested by the client machine. Once the server has selected a target machine for the task, the task file is copied to the selected target machine. The target machine selects a task to be performed based on this priority information contained in the task file copied to the target machine. Once a task is completed, the results are copied back to the server which compares them to a set of "golden results" and creates a comparison report which is mailed back to the user that requested the test.
One disadvantage of the system proposed by Sun Microsystems is that the system cannot be easily adapted or expanded because the design of the system is based on the concept of pre-allocating tests to test machines having the least current load. The central server pre-schedules tasks on test machines and then sends the task to the test machine on which the task is scheduled to be performed, even though the test machine may be currently executing another task or have other tasks already scheduled on it. This scheduling method prevents the system from automatically adapting and re-scheduling an already-scheduled task if a new test machine is added to the test machine pool because the task, once scheduled, is sent to the test machine. Furthermore, if a task fails after performance of the task has begun, or if a test machine fails after tasks have been scheduled on it, all of the affected tasks will need to be rescheduled and no provision is made for automatically rescheduling the tasks.
Accordingly, a need exists for a method and apparatus for scheduling jobs in an automated testing system which renders the automated testing system adaptive and easily expandable and which schedules jobs in such a manner that efficiency in job allocation and performance is maximized.